Factors Mediating the Success of Observation-based Problem Solving

Studies of complex dynamic control tasks (CDCTs) have revealed that when problem solvers learn about a system indirectly through observation (Berry, 1991; Lee, 1995), their procedural knowledge of the system is impaired relative to their declarative knowledge. However, when learning is through direct interactions with the system, then problem solvers declarative knowledge is impaired relative to their procedural knowledge. Osman (in press) claims that one reason that observation based learning produces such poor procedural knowledge is that observers are prevented from hypothesis testing during learning and monitoring the status of their knowledge of the system; this has been shown to be critical in the acquisition and application of relevant knowledge in CDCTs (e.g., Sanderson, 1989). The present study explored the effects of preventing and encouraging hypothesis testing during observational learning on the accuracy of declarative and procedural knowledge of a dynamic problem solving task. The findings show that when instructions promote self evaluative processes during hypothesis testing, problem solving ability is improved compared to when self evaluative processes are prevented. Conflicting theoretical issues CDCTs have been a popular task environment for examining many phenomena, including motivational and affective processes in complex decision making (e.g., Locke & Latham, 2002), skill learning in naturalistic decision making (e.g., Brehmer, 1992) memory and attentional processes in problem solving (Burns & Vollmeyer, 2002), and implicit learning (e.g., Berry, 1991; Lee, 1995). Their popularity and range make them ideal for studying the acquisition and transfer of skill-based knowledge in a variety of complex interactive environments. Declarative vs. procedural skill learning: CDCTs have been described as procedural tasks because they involve perceptual-motor behaviors that fulfill a set of constraints in order to achieve a goal. Procedural learning involves controlling an environment that is dynamic: i.e., it is changing as a consequence of the learner’s actions. The knowledge that is acquired is procedural, and represents “knowing how” to perform actions that are tied to specific goals. This is different from declarative knowledge, which is “knowing that” of particular facts about the underlying actions and structural knowledge concerned with the goal itself (e.g. Anderson, 1982). The popular position on procedural learning in problem solving is that procedural knowledge and declarative knowledge are dissociated (Berry, 1991; Berry & Broadbent, 1984; Dienes & Berry, 1997; Lee, 1995), and that they are supported by functionally separate cognitive mechanisms (e.g., Squire, 1986). It follows that having declarative knowledge alone will impair one’s later ability to perform a procedural task. One method used to demonstrate this involves training people on a procedural task by observing another perform it first: Because the learners are explicitly monitoring what they are observing, this is claimed to generate declarative knowledge (e.g., Kelly & Burton, 2001). Berry (1991) and Lee (1995) used this method to compare the effects of procedural-based and observation-based learning. They showed that, when participants later came to problem solve, the observers’ ability to perform the procedural task was poorer than that of procedural-based learners. Returning to the example, the suggestion here is that John’s ability to master the device would have been more successfully achieved had he tried to learn-by-doing rather than learn-byobserving. Although popular, the claim that proceduralbased learning has an advantage over observation-based learning in problem solving has attracted little empirical support (e.g., Berry, 1991; Lee, 1995). Goal specificity: Some (e.g., Burns & Vollmeyer, 2002; Sweller, 1988; Vollmeyer et al., 1996) claim that during learning, the nature of CDCT encourages people to generate hypotheses (i.e., they select which inputs they want to manipulate) and test them (i.e., they make predictions as to which outputs the inputs changed will effect, and the outcome of the manipulation is monitored and used to update the model of the system). Evidence for this comes from studies that compare different types of goal instructions during learning. For instance, instructions like “explore the system”, an example of non-specific goal, are contrasted with “learn about the system while trying to reach and maintain specific outcomes”, an example of a specific goal. In the control test phase, specific goal learners perform more poorly than non-specific goal learners. Burns and Vollmeyer’s (2002) recent account of the differential effects of goal specificity on problem solving develops on Dual-Space theory (Klahr & Dunbar, 1988). The theory proposes that a skill is acquired by using the same principles that underlie scientific discovery: i.e., designing the appropriate procedure (experimental design) to evaluate a theory (hypothesis formation). Dual-Space theory deconstructs a task into spaces: the hypothesis space, which consists of the hypotheses generated, and the experimental space, which consists of instances of the problem that can potentially be tested. Search in the hypothesis space is guided by self-evaluative processes and